Restructuring your QLD: rebutting allegations of 'dumbing down'

Chris Gale, Leeds Metropolitan University

Chris’ paper reflects on the structure and content of the law curriculum and the development of appropriate strategies for learning, teaching and assessment, as well as opportunities for staff development, in the context of a revalidation of qualifying law degrees at Leeds Law School. It updates the paper Chris presented at LILI 2002, entitled Revalidation, validity and the tensions between.

The QLDs (qualifying law degrees) in Leeds Law School fell due for revalidation in 2001-02. In examining our provision we were aware that many students failed the two double modules that were assessed only by exam at the end of the year. University regulations were such that these students thus failed the year and, while they could enrol on Year 1 from scratch again the following September, many did not and were lost to the university. On further investigation, it became apparent that many students who fell into this category were either mature students returning to education after a break or students who had no family tradition of higher education, many of whom came from addresses in post code band D. Many of these same students had performed adequately or better in the other two Year 1 double modules, assessed by assignments alone or by a combination of group project and exam.

Drivers such as widening participation and retention goals made the Scheme Development Team (SDT) consider whether introducing alternative methods of assessment could retain the necessary ‘quality’ in the modules and also give students an opportunity to show they met level/module outcomes by means of other assessment activities. If this could be achieved, it would clearly help meet university retention targets, but would equally clearly help cut the awful waste in individual human terms of a student, who could become a capable lawyer, leaving after Year 1 marked effectively as a failure.

In examining these matters the SDT was interested to note how it was treated by some as looking for a way to ‘dumb down’ the degree to help retention rates at the cost of ‘quality’. It was also impressed by the thought that had it not been for the drivers mentioned above no serious action may have been considered, leaving the waste issue untouched and maybe contributing to further accusations of elitism among lawyers.

At the same time the potential means of addressing the problem opened up possibilities for more interesting and possibly more appropriate means of assessing. We were concerned that we were maybe 15 years behind colleagues elsewhere in the country, but our validation panel assured us that while some had moved matters forward, many other institutions were even further away from grappling with the issues than we were.

This is an account of what happened in one law school. We do not claim to be ‘ahead of the game’ by any means – maybe just the opposite. In many ways it is probably an account of what should have happened some time ago, not in 2001/2002. It does however show how reflecting – for various reasons not always welcomed – on provision can lead to results that should be beneficial to all.

History

Although we have been running a Law with Information Technology degree since 1998 as well as the LLB, this paper will concentrate on our LLB provision. That was our only undergraduate course at the time of the last major revalidation in 1996, and thus the comparisons, comments and figures will be both simpler if only the one course is considered and also cover a more meaningful period of time. For simplicity, I will refer to the degree structure in place 1996-2002 as the ‘old provision’ and the one designed to run from October 2002 as the ‘new provision’.

At Level 1 of the old provision, students studied double modules (30 credit points each) of Law and Legal Skills, Public Law, Contract and Tort. At Level 2, double modules in Criminal Law, Property Law, Trusts and European Law were studied and Options totalling 120 credits were studied at Level 3. Full time students were supposed to complete one level each academic year and part time students studied modules worth 90 credit points each year, meaning that they were scheduled to complete their studies in four years. Part time students studied Law and Legal Skills, Contract and Tort in Year 1, Public Law, Criminal Law and European Law in Year 2, Property Law, Trusts and one double module Option in Year 3, and 90 further credit points worth of Options in Year 4.

The perceived problem

In examining the old provision in readiness for revalidation, we were aware that many students failed the two double modules that were assessed only by exam at the end of the year, namely Contract and Tort. In truth, the exact numbers surprised us, but the general trend had been known for some while. University regulations were such that these students thus failed the year and, while they could enrol on Year 1 from scratch again the following autumn, many did not and were lost to the university.

On further investigation, it became apparent that many students who fell into this category were either mature students returning to education after a break or were students who had no family tradition of higher education, many of whom came from addresses in post code band D. Many of these same students had performed adequately or better in the other two Year 1 double modules which were assessed by assignments alone (Law and Legal Skills) or by a combination of group project and exam (Public Law). We were also aware that of this group of ‘failing students’, those who did enrol again on Year 1 often just managed to overcome the ‘double exam’ hurdle at the second time of trying, but then progressed well at Levels 2 and 3 (where all modules were assessed by a combination of written assignment and exam or by written assignment alone), frequently attaining 2:2 or 2:1 awards. Perhaps some of these individuals were just not very good at examinations – as could be the case with any group – but it seems more likely that most were just ‘getting back into the routine’ and by level 2 were taking it in their stride.

Figure 1: students failing both Contract and Tort exams but being successful in other modules, 1996-2002

	1997	1998	1999	2000	2001	2002
FT	16/85	20/96	13/100	18/105	22/111	25/122
PT	14/42	12/36	8/45	21/49	11/27	11/38

Figure 2: approx % of ‘adult returners’ or students with no family HE tradition of the students in figure 1

	1997	1998	1999	2000	2001	2002
FT	75	75	100	50	80	80
PT	92	100	100	100	82	100

Figure 3: students who, having failed as above, returned to start the degree again from scratch  

	1998 (failed in 1997)	1999	2000	2001	2002
FT	8	10	3	11	10
PT	4	5	7	11	6

Figure 4: final degree classifications of those who passed Level 1 second time round

	2001		2002
FT	2:1×2	2:2×5	3×1(8)	2:1×4
PT	2:2×2	3×2 (8)	2:1×1	2:2×2 (3)

Why had Contract and Tort been assessed in the way it was in the old provision? It clearly was out of line with all other subjects, be they Foundations or not. Looking back on old minutes from 1996 it was apparent that the then module leaders, both former practitioners who have since retired, were of the opinion that the only true test of legal knowledge was by end examination. The contents of their modules and the means of assessment had not been questioned in 1996, and any attempts to review it in the light of the results mentioned above had been resisted. Prior to 1996, many more modules were assessed purely by examination, but after that date Contract and Tort stood alone – and both at Level 1.

The process

Following my appointment as Head of Undergraduate Studies in 2001 to pave the way for the revalidation, the critical review of the old provision took into account drivers such as widening participation and retention, which had had little or no impact and had certainly had less significance in 1996, and made the Scheme Development Team (SDT) consider whether alternative methods of assessment could still retain the necessary ‘quality’ in the Contract and Tort modules but also give students an opportunity to show they met level/module outcomes by means of other assessment activities. If this could be achieved, it would clearly help meet university retention targets, but would also arguably help cut the awful waste in individual human terms of a student who could be a capable lawyer leaving after Year 1 marked effectively as a failure. Although the statistics reported above had been recognised in each annual course report, they had been rationalised as representing students who were ‘not suited to law’. The underlying assumption that they were destined to fail somewhere along the academic line and that Level 1 was as good a place as any had never been seriously challenged. Clearly there are some students who do fit into this category (and even they could be dealt with more sympathetically!), but the results of the ‘returners’ who did eventually overcome the hurdle at Level 1 suggests that a lot of potentially good students were being ‘thrown away’ for no sound reason.

In examining these matters, the SDT was interested to note how it was treated by some colleagues as looking for a way to ‘dumb down’ the degree to help retention rates at the cost of ‘quality’. It was also impressed by the thought that had it not been for the drivers mentioned above, no serious action may have been considered, leaving the waste issue untouched and maybe helping to lead to further accusations of elitism among lawyers and universities. At the same time, the potential means of addressing the problem opened up possibilities for more interesting and possibly more appropriate means of assessing. We were concerned that we were maybe 15 years behind colleagues elsewhere in the country, but our validation panel assured us that while some had moved matters forward, many other institutions were even further away from grappling with the issues than we were.

It is interesting to observe that some colleagues felt – and some maybe still feel – that anything other than an unseen examination is not a ‘proper assessment’ and any results coming from it thus devalued. Most of these people were themselves at university when this was virtually the only means of assessment and, if they qualified as solicitors under the old Law Society Part II route, this assessment only by examination would again be the case. The Law Society itself recognised in the late 1970s that ‘rote learning’ was not the only way of assessing competence in a subject and, by observation, it seems clear that even those who could rote learn for the Part IIs did not always understand a subject or were unable to work with its underlying concepts and philosophies as soon as they leave the examination hall. In short, many made poor practitioners, even though they carried the ‘passing of examinations’ badge. So, at best, examinations are one way of assessing the competence of some of the candidates; others can hide their lack of comprehension through slavish adhenece to model answers or some such, and a number who did understand the underlying subject matter but were just unable to bring it to bear in examination conditions were being written off as failures. At the very least, the SDT asked module teams to consider other methods of assessment – and, if they chose not to, to be able to defend any future anomalies in their pass rates to the appropriate authorities!

It is probably correct to say that incidents of plagiarism are on the increase, and the temptation is to go back to an examination as being one of the methods of assessment less likely to be tainted. That said, if this disadvantages potentially good students, it cannot be the correct response – our task surely is to devise fair and appropriate assessment and to be able to factor out plagiarism as much as possible – or at least to be able to detect it when it does occur. If anyone has the time or resources to do that for all scenarios, please let not just me but all law schools know – you will be on to a winner!

The solution (or, at least what we did)

In line with university policy, all teaching in the new provision is in single module (15 credit points) chunks. Contract and Tort simply have two single modules and Contract 1 and Tort 1 run in Semester 1 and Contract 2 and Tort 2 in Semester 2, so the material to be covered could remain the same and the time for covering it would also remain. Assessment of both Contract 1 and Tort 1 is by means of written assignment, given to the students in approximately week 7 of the semester with an answer handed in at the end of week 15. Assessment of Contract 2 is by unseen examination and Tort 2 by a ‘scenario’ given to students some time before they collect the questions to be asked on it which they have to complete within two working days once given, although they may use such resources as they like in producing their answers.

How did the Contract and Tort module teams come to accept this? Probably a number of factors helped. First, the module leaders from 1996 had retired. Second, no one wants to run a module with the worst success rate on the course, and with the figures shown above the SDT was able to put this forward – the suggestion that the subject matter was not so inherently difficult as to justify (or even explain!) such a failure rate was accepted without demur on production of the figures. And finally, all the other double modules that had moved to being assessed by means of an assignment and an examination in 1996 had seen their initial pass rate go up and no discernible decrease in ‘quality’ – at least not as encountered by external examiners or the internal university processes.

Conclusions

It is, of course, too soon to say if any of the developments discussed above have the results suggested – of disadvantaging the least number of students and of being such a mix assessment formats that all candidates encounter a format to match their ability and enable them to succeed. Perhaps a future LILI Conference could have the benefit of that data, one, two and/or three years down the line! What we do know is that the ‘returners’ to Level 1 who were unsuccessful in Contract and Tort in 2002 are unanimous in saying that they feel more comfortable with the new system – exams have a place, but the result of failing would not be as disastrous as previously, as up to three single modules may be retaken without having to start again from scratch. From a personal point of view, I feel that having questioned the previous structure successfully what we have in place is better, but rather unimaginative. We have, in the current academic year, set up a working party to consider the suitability of methods of assessment not considered – or rejected on limited evidence – by module teams in 2002. UKCLE is providing a roadshow later in the academic year to complement our ‘in-house’ suggestions, with the hope that a broader based diet of assessment can be devised and implemented in the medium term.

The second (and final) thought I have is this. If it were not for the process of review (often derided among lawyers), the problem may not have been ascertained (or at least documented) in the way it was. If the ‘R’ and ‘WP’ words (again, often derided among lawyers although increasingly grudgingly accepted!) were not as much to the fore as they are, much of this process might not have been engaged. If any benefit is brought about to any student -and I am sure that it will – then these processes seem to take on a ‘proper meaning’ in themselves, rather than being the ‘outcrops of an interfering bureaucracy’ (as often argued by lawyers). We frequently bemoan lack of money and time in law schools – these of themselves would solve little. Added to the ‘quality stimuli’ and the other drivers mentioned in this paper, we could – for probably less than we think – have the Rolls Royce academic provision that would please government, universities, students and professional bodies – Retention and Widening Participation may not be just drivers for their own sakes, but may be helping improve the quality – and rationality – of provision and experience for all students.

Last Modified: 12 July 2010